Skip to content

Experimental: Use ActiveRecord models rather than JSONSchema to model block types#553

Draft
edavey wants to merge 22 commits into
mainfrom
exp/use-ar-rather-than-json-schema
Draft

Experimental: Use ActiveRecord models rather than JSONSchema to model block types#553
edavey wants to merge 22 commits into
mainfrom
exp/use-ar-rather-than-json-schema

Conversation

@edavey
Copy link
Copy Markdown
Collaborator

@edavey edavey commented Mar 30, 2026

See the ADR (which is the first commit) for an overview of this experiment:


11. Experiment with Traditional ActiveRecord Models for Content Blocks

Date: 2026-03-11

Status: Experimental

Context

Current State

The Content Block Manager currently uses a schema-driven generic approach for modeling content blocks:

  1. Single Document Model: All block types use a single Document model (no subclasses).
  2. Single Edition Model: All block types use a single Edition model with a generic details JSON column.
  3. JSON Schema Validation: Block type structure is defined and validated via JSON schemas stored in app/models/schema/definitions/*.json.
  4. Generic Forms: Form interfaces are dynamically generated from the JSON schemas.

This architecture provides flexibility and allows new block types to be added by defining a JSON schema without creating new models or migrations.

This was a necessary technical design initially as all schemas were defined in Publishing API. However, in order to have greater control, particularly over validation, we've recently moved the schema into this repository (See [ADR 10 Make Content Block Manager the source of truth for schemas][])

The use of JSON schema definition is no longer a requirement.

The Question

As the application grows, we're evaluating whether the schema-driven approach provides the best developer experience and maintainability, or whether a traditional Rails ActiveRecord approach with dedicated models and tables might be clearer and easier to work with.

Specifically, we want to understand:

  1. Developer Experience: Is it easier and faster to work with explicit Rails models and associations vs. generic JSON columns? Our experience when adding a new block type is that there are numerous differences or additional features which are needed. At face value they appear minor but in reality prove complex to introduce to an application model which is entirely generic. Our small number of classes can do everything, but at what price in terms of complexity and cognitive load.
  2. Maintainability: Are dedicated models with typed columns easier to understand and modify than schema definitions?
  3. Adding Fields: Is it simpler to add a migration for a new column vs. updating a JSON schema?
  4. Performance: Do typed columns with proper indexes perform better than querying within JSON?
  5. Testing: Are traditional model tests more straightforward than testing schema validation?

Decision

We will conduct an experiment by implementing an alternative architecture using traditional ActiveRecord models, running in parallel with the existing schema-driven system.

Experiment Scope

We will implement the TimePeriod block type as a proof-of-concept using the new architecture. This will allow us to:

  • Compare both approaches side-by-side in the same codebase
  • Evaluate real-world developer experience
  • Make an informed decision about which direction to pursue

This is not yet a decision to migrate away from the existing schema-based paradigm - it is purely exploratory.


Current state

current_state

Proposed target state

proposed_target_state

To try this out locally you can either:

1. Use some conditionally applied UI elements

See some conditional UI elements by giving yourself the schemaless_experiment permission:

User.last.permissions << `schemaless_experiment`
User.last.save

or by adding ?schemaless_experiment=true to your URL

2. Navigate manually to the new-style TimePeriod form

/block/time_period_editions/new

new_time_period_edition_form

Screenshot of new style TimePeriod

new_style_time_period

edavey added 5 commits March 30, 2026 12:35
We make this a @wip as we don't expect it to pass for a while yet. But
it's nevertheless helpful to state our anticipated user-journey clearly
at the outset.

This is inline with the Outside->In development method.

See https://www.slideshare.net/slideshow/outsidein-development-with-cucumber-and-rspec/1221433
Add `block_documents` table to support the experimental `Block::` namespace
architecture for content blocks. This table serves as the container for
content block documents, with `block_type` discriminator.
Add `block_editions` table using Single Table Inheritance (STI) to support
different edition types (`TimePeriodEdition`, `TaxEdition`, etc.).

It:

- uses STI via 'type' column (unlike `block_documents` which uses block_type)
- includes common fields (`title`, `description`, `instructions_to_publishers`) on base table
- does NOT include 'details' JSON column - content stored in type-specific tables instead
- no 'state' column - workflow state machine can be added 'later', this is just
a proof of concept
Adds content table for `TimePeriodEdition#date range` data. This table
stores the `start` and `end` datetime values for time period blocks.

Note:

- Uses `datetime` columns (not separate date/time) for simpler storage
  as we know:

  - (from experience) that validation is easiest on a complete
   "date time"

  - we need a range of representations of a `TimePeriod#date_range`, e.g
   "2027-2028", "April 2027 to April 2028", "April" etc. Supporting these
   formats is more simple using single `start` / `end` values.

- Unique index on `edition_id` enforces `has_one` relationship

- Both `start` and `end` are required (not null)

- Note that validation that `end > start` will be handled at the model level
@edavey edavey force-pushed the exp/use-ar-rather-than-json-schema branch from 9b15e4b to d47979e Compare March 30, 2026 12:40
edavey added 17 commits March 30, 2026 14:02
Creates the `Block::Document` model as a concrete class (no STI) that serves
as the container for content block documents.

Note:

- Uses `block_type` string column instead of STI to track edition type
- Auto-generates content_id (UUID) and embed_code on creation
- Provides embed code generation methods for block and field-level codes
Creates Block::Edition as an abstract STI base class for all edition types
(TimePeriodEdition, TaxEdition, etc.).

Note:

- Abstract class - cannot be instantiated directly
- STI via 'type' column in block_editions table
- Belongs_to :document relationship to Block::Document
- Common validations: title presence
- Abstract #details method that sub-classes must implement
- No workflow state machine (this will follow "later")
Creates Block::TimePeriodEdition as a concrete STI subclass of Block::Edition.
This is a minimal implementation - associations and the #details method will
be added in following commits.

Note:

- Inherits from `Block::Edition` via STI
- Type column stores "Block::TimePeriodEdition"
- Inherits title validations from parent
- No date_range association yet (added in following commit)
- No #details implementation yet (added in following commit)
Creates the content model for storing time period date ranges. This model
stores the `start` and `end` datetime values and provides serialisation to the
details hash format.

Key features:

- Belongs_to Block::TimePeriodEdition (enforced with unique index in DB)

- Validates presence of `start` and `end` andthat `end` must be after `start`

- `#to_details` method serializes to hash with formatted date/time strings

- Uses `datetime` columns for simple storage and validation (in contrast to
  the current main implementation which uses separate `date` and `time` fields.
  We expect the main current implementation to switch to using a single datetime field
  for `start` and for `end`)
Builds the `TimePeriodEdition` model by adding the `date_range` association
and implementing the `#details` method required by the `Edition` base class.

Note:

- `has_one :date_range` association
- `accepts_nested_attributes_for :date_range` (for form handling)
- `#details` method composes `description` + `date_range.to_details`
- Uses `.compact` to handle nil `description` or missing `date_range` gracefully
- Add `has_many :time_period_editions` scoped association to Document
- Add `inverse_of: :document` to associations for proper bidirectional
  association caching
- Create `Block::OtherEdition` stub class for testing STI scoping

This makes the API cleaner and more idiomatic - each block type
gets its own first-class association rather than manually specifying
the STI type parameter.
Add routes and skeleton controller for managing TimePeriodEdition
resources as the first step in a two-step block creation workflow:

1. TimePeriodEditionsController - create/edit the edition (common fields)
2. TimePeriodDateRangesController - supply/edit the type-specific date range

Routes: GET/POST /block/time_period_editions, plus show/edit/update
Model changes:
- Include HasLeadOrganisation module for lead_organisation_id validation
- Add alphanumeric title validation (must contain letter or number)
- Add before_validation callback to set document.sluggable_string from title

I18n:
- Add block/edition error messages for title and lead_organisation_id

Tests:
- Unit tests for alphanumeric title validation
- Unit tests for lead_organisation_id presence validation
The first step in the two-step block creation workflow:

Controller:
- new/create actions for creating TimePeriodEdition with associated Document
- show action for viewing created edition
- Strong parameters for edition fields
- Organisation dropdown population

Views:
- Form partial with title, description, lead organisation, instructions
- Error handling with GOV.UK error summary component
- New and show templates
Organisations use UUIDs, so the lead_organisation_id column needs to be
a UUID rather than an integer for the form to actually persist the value.

- Add migration to change column type from integer to uuid
- Include ::Edition::HasLeadOrganisation module which provides:
  - OrganisationValidator (validates the organisation exists)
  - lead_organisation method to fetch the org
- Remove manual presence validation (now handled by the module)
Add routes and skeleton controller for managing TimePeriodDateRange
as the second step in the two-step block creation workflow.

Routes are nested under documents:
  GET/PATCH /block/documents/:document_id/time-period-date-ranges/:id

The edition's date ranges are managed through the edition's nested attributes.
Full implementation of the second step in the two-step block creation
workflow - managing the date range for a TimePeriodEdition.

Controller:
- before_action filters for document and edition lookup
- edit/update actions for modifying date ranges via nested attributes
- show action for viewing the completed block

Views:
- DateTime form with GOV.UK date input and time select components
- Start and end datetime fields with separate date/time inputs
- Show page with summary list displaying block details
- Edit page with back link navigation

Includes request specs covering success and validation error paths.
After creating a TimePeriodEdition (step 1), redirect the user to the
date range edit form (step 2) to complete the time period details.

This completes the two-step UX workflow:
1. Create edition with common fields (title, description, organisation)
2. Supply type-specific date range
After saving the date range (step 2), redirect to the edition show page
which displays the complete block with all its details:

- Edition fields: title, description, instructions, embed code
- Date range fields: start date, end date (formatted via DatePresenter)

Adds a cucumber feature describing validation and removes @wip tag -
both scenarios now pass.
At present it's not possible to navigate to view or create new-style experimental
TimePeriodEdition blocks via the existing UI.

To add some conditional UI elements we add a new `schemaless_experiment` permission
which we'll used to offer navigation to our experimental UI views.
We add some conveniences to make it easy to see and use the experiment
in making a (Time period) content block, without use of the generic and
complex schema-based code.

If the user has the 'schemaless_experiment' permission or appends the
`?schemaless_experiment=true` url param they will:

- see a header nav element "New style blocks" linking to a list of
`Block::TimePeriodEdition`s
- see a 'Create new-style block' CTA on the homepage linking to the new
time period edition form
Include DateValidation concern to handle invalid multiparameter date
assignment (e.g. month=23) gracefully with validation errors instead
of raising ActiveRecord::MultiparameterAssignmentErrors.
@edavey edavey force-pushed the exp/use-ar-rather-than-json-schema branch from d47979e to 8af4cd2 Compare March 30, 2026 13:03
@edavey edavey requested review from graycodes and pezholio March 30, 2026 13:07
@pezholio
Copy link
Copy Markdown
Contributor

Thanks for this - I haven't had a proper look yet, but I do worry that us having separate controllers etc for each block type is a regressive step, and is a bit out of step with the work that ie Whitehall are doing with config driven document types (see https://github.com/alphagov/whitehall/blob/main/docs/adr/0006-config-driven-content-types.md) I wonder if there's something we can learn from their approach?

@pezholio
Copy link
Copy Markdown
Contributor

Been thinking about this a bit more - is there a way we can define the steps explicitly in config, so we can avoid having to guess them based on the shape of the schema?

@pezholio
Copy link
Copy Markdown
Contributor

More thoughts! I think I'd be happy giving this a go, but I'd rather the unit of difference was just the model, rather than having separate views and controllers etc. With that, we can define validations and data manipulation in the model, as well as potentially the workflow shape too.

validates :end, presence: true
validate :end_date_after_start_date

def to_details
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should be consistent with other publishing apps and use presenters to present information we send to the Publishing API - example here https://github.com/alphagov/whitehall/blob/main/app/presenters/publishing_api/case_study_presenter.rb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants